As the number of computing devices increase across society, electronic data management has become increasingly challenging. Modern devices create and use ever increasing amounts of electronic data ranging from digital photos and videos, to large data sets related to any number of topics including energy exploration, human resources, seismic activity, and gene research. This explosion in digital data has naturally led to ever increasingly large amounts of data that must be stored. Correspondingly, the data storage field is under constant pressure to increase size, performance, accessibility, reliability, security, and efficiency of data storage systems.
In order to meet these demands for data storage, various storage systems have been developed. Large scale storage systems often include storage appliances that include arrays of spinning hard drives, magnetic tape drives, and/or solid state drives. Multiple storage appliances may be networked together to form a cluster. A cluster of storage appliances provides for added capacity as well as added redundancy, as compared to a single appliance. Storage appliances in a cluster may be configured to synchronize data between the clustered appliances to increase both access speeds and availability.
Referring to FIG. 1, an example storage network 100 is depicted. This example storage network includes one or more of Oracle's ZFS storage appliances 110, 120 each including one or more disk drives. As mentioned, the storage appliances may be clustered; therefore, a second storage appliance 120 may be configured to sync with a first storage appliance 110. The storage network 100 is accessible by clients 130, 132, 134, 136 using a network 140. Generally speaking, the storage appliance 110, 120 manages the storage of data on spinning disk or solid state hard drives. The depicted networks may be local in nature or geographically dispersed such as with large private enterprise networks or the Internet.
Although discussed in the context of ZFS appliances, the storage appliances 110, 120 may include any conventional storage appliance. ZFS is a combined file system and volume manager designed by Sun Microsystems® in 2005 that allows for data integrity verification and repair, high storage capacities, along with numerous other features. ZFS based systems utilize storage pools (often referred to, as zpools) constructed of virtual devices (often referred to as vdevs) constructed of block devices. A block device is any device that moves data in the form of blocks including hard disk drives and flash drives. A ZFS volume is a dataset that represents a single block device. A virtual device (vdev) may span a number of block devices/volumes and a zpool may include one or more vdevs.
One feature of computing systems running ZFS, as well as other file systems, is the ability to perform replication. Replication allows for a first system to create a replica of itself or a portion of itself on another system by utilizing snapshots. One advantage of replication is that it allows for the second system to be regularly synchronized to the first system. For example, the systems may be configured to perform replication at certain times thereby keeping the systems relatively synchronized. This is accomplished by taking snapshots of the various file systems, volumes, or datasets on the first network storage appliance that a user wishes to replicate and then transferring the snapshots and any associated data to the second network storage appliance. Although the process of creating a snapshot on the first network storage appliance is very quick, the replication process may take a significant amount of time, sometimes taking many hours to complete, depending on the size of the data to be replicated.
Currently systems suffer from not being able to accurately monitor the progress of replication, or determining that the replication has encountered a problem causing the replication to hang or fail entirely. Because of this, an administrator may suspect a problem had occurred and needlessly restart the replication causing replications to take even longer than required and causing user dissatisfaction among other problems.
It is with these and other issues in mind that various aspects of the present disclosure were developed.